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SUMMARY 


Problem 


The  major  purpose  of  this  project  is  to  apply  modern  instructional 
and  computer  technology  to  the  management  of  large-scale  instruc- 
tion, while  minimizing  in-line  computer  and  material-preparation 
costs.  A Computer-Assisted  Instruction  Study-Management  System 
designed  under  a previous  project  used  computer-programmed  questions 
to  maintain  student  attention.  This  project  is  an  expansion  of  that 
earlier  work  to  integrate  books,  computers,  and  live  instructors  in 
a Course-Management  System  (CMS).  CMS  is  intended  for  use  in 
courses  with  a large  number  of  students  and  instructors. 

Approach 

The  logistic  problems  associated  with  multifaceted  instructional 
programs,  large  enrollments,  and  many  instructors  were  solved,  in 
part,  by  using  PLATO,  a computer-assisted  instruction  system 
developed  by  the  University  of  Illinois.  Students  in  the  computer- 
managed  system  acquire  basic  information  primarily  from  reading. 

Their  attention  to  the  course  material  is  maintained  and  their 
progress  through  the  material  is  monitored  by  a previously  developed 
Computer-Assisted  Instruction  Study-Management  System.  Lectures  and 
standard  quiz  sections  are  little  used.  The  time  of  instructors  is 
vested  in  remediation  for  students  having  trouble  mastering  the  core 
curriculum  and  in  teaching  seminars  and  special  projects.  The 
computer  is  used  to  manage  study  behavior  by  administering  on-line 
achievement  tests,  scheduling  group  tutorial  and  seminar  sessions, 
and  making  lesson  assignments.  The  course-management  system  features 
mastery  learning,  and  permits  able,  hardworking  students  to  complete 
the  course  in  much  less  than  the  usual  time  period. 

Conclusion 


The  Course-Management  System  was  developed  and  tried  out  in  an 
introductory  college  economics  course  with  an  enrollment  of  360 
students.  A comparison  with  student  performance  data  from  prior 
years  shows  that  present  students  have  learned  as  well  and  are 
favorably  disposed  toward  the  CMS. 
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INTRODUCTION 


The  Course  Management  System  (CMS)  was  designed  to  integrate  books, 
computers,  and  live  teachers  in  an  effective  manner.  The  system  is 
intended  for  courses  with  large  numbers  of  students  and  instructors, 
such  as  introductory  courses  in  community  colleges,  universities  or 
military  settings.  The  logistic  problems  associated  with  multifaceted 
instructional  programs  and  large  numbers  of  students  and  faculty  were 
solved,  in  part,  by  using  PLATO — a computer-assisted  instruction  system 
centered  at  the  University  of  Illinois. 

In  CMS,  students  are  expected  to  acquire  basic  information  and  con- 
cepts primarily  from  individual  reading.  Their  attention  to  the  material 
is  maintained  and  their  progress  monitored  by  a previously  developed  and 
evaluated  Computer-Assisted  Instruction  Study-Management  System  (T. 
Anderson  et  al. , 1974a,  1974b).  Lectures  and  standard  quiz  sections  are 
little  used.  The  time  instructors  normally  spend  on  routine  lectures 
and  reviews  is  invested  in  remediation  for  students  having  trouble 
mastering  the  core  curriculum,  and  in  teaching  a variety  of  topical, 
high-interest,  activity-oriented  seminars,  and  special  projects.  The 
role  of  the  computer  is  to  manage  study  behavior,  administer  on-line 
achievement  tests,  schedule  group  tutorial  and  seminar  sessions,  and 
conduct  other  functions.  CMS  features  mastery  learning,  and  permits 
able,  hard-working  students  either  to  complete  the  course  in  as  little 
as  12  weeks,  or  to  gain  extra  credit  by  taking  additional  seminars. 


DESCRIPTION  OF  THE  COURSE-MANAGEMENT  SYSTEM 
Role  of  the  Computer-Assisted  Instruction  Study-Management  System 

The  core  of  the  Course  Management  System  is  the  Computer-Assisted 
Instruction  Study-Management  System  (CAISMS) . Students  often  fail  to 
learn  from  text  because  they  do  not  study  it  thoroughly  enough,  and 
CAISMS  is  designed  to  increase  attention.  A range  of  evidence  shows 
that  procedures  that  induce  meaningful  processing  of  text  facilitate 
learning  (e.g.,  Barclay,  1973),  and  75  years  of  research  indicate  that 
an  effective  way  of  managing  processing  activities  is  to  ask  questions 
about  what  has  been  read  (R.  Anderson  & Biddle,  1975).  This  is  the 
technique  used  in  CAISMS. 

Under  CAISMS,  the  student  signs  in  at  a PLATO  terminal  and  receives 
a brief  study  assignment.  Upon  completing  the  assignment  in  a nearby 
work  space,  the  student  again  signs  onto  the  terminal.  This  time  he 
receives  a short  quiz  on  the  assignment  just  completed.  If  the  quiz 
performance  is  below  75%  correct,  he  reviews  the  same  study  assignment. 
If  performance  is  above  75%  correct,  he  receives  a new  assignment.  The 
quizzes  are  not  graded,  but  the  student  must  pass  a quota  of  them  to 
become  eligible  for  graded  examinations. 
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The  quiz  items  require  comprehension  of  the  important  concepts  in 
the  text  as  judged  by  economists  on  the  project  team.  Each  item  either 
paraphrases  the  language  in  the  text,  or  calls  for  application  of  con- 
cepts and  principles  to  examples  different  from  any  appearing  in  the 
text.  These  types  of  items  increase  the  probability  that  the  student 
will  engage  in  meaningful  cognitive  processing  (cf.  Watts  & R.  Anderson, 
1971;  Felker , 1974). 

The  CAISMS  was  evaluated  in  an  experimental  study  in  which  a 
randomly  selected  group  of  students  used  it  for  a semester  and  a 
randomly  selected  control  group  did  not.  Both  groups  used  the  same 
text,  had  the  same  instructors,  and  took  the  same  hourly  and  final 
examinations.  Results  indicated  that  the  CAISMS  significantly  improved 
student  achievement  and  attitudes.  It  should  be  emphasized  that  this 
study  did  not  investigate  the  effects  of  replacing  lectures,  but  rather 
of  complementing  lectures  with  CAISMS.  Even  In  an  adjunct  role,  CAISMS 
was  powerful  enough  to  increase  achievement. 

Currently,  300  students  at  the  University  of  Illinois  are  using  CAISMS 
as  a part  of  the  Course-Management  System.  Data  from  the  first  achieve- 
ment test  support  the  contention  that  students  do  as  well  using  CAISMS 
as  when  they  are  required  to  attend  lectures  and  to  read  the  text.  Test 
means  show  that  students  using  CAISMS  this  semester  are  performing 
approximately  as  well  as  students  did  in  previous  semesters  where  the 
lecture-discussion  method  was  used. 

CAISMS  does  not  only  manage  the  students1  independent  study  of  the 
text  in  the  Course-Management  System,  but  it  also  replaces  traditional 
lectures  and  routine  review  sessions.  These  lectures  and  reviews  are 
typically  dull,  ineffective,  and  inefficient  (cf.  Lloyd  jet  al. , 1972). 

They  squander  human  resources.  It  has  been  clearly  established  that  a 
course  can  be  successful  without  lectures.  For  example,  the  June  1972 
issue  of  the  Personalized  System  of  Instruction  Newsletter  reported  a 
questionnaire  study  of  190  courses  using  self-instructional  materials. 

Most  of  the  course  plans  had  entirely  eliminated  lectures.  The  disciplines 
most  heavily  represented  were  psychology,  physics,  engineering,  mathematics, 
chemistry,  and  biology,  with  a smattering  of  courses  from  such  fields  as 
Spanish,  English,  and  sociology.  Almost  all  respondents  claimed  they 
learned  more  from  personalized  (i.e.,  individualized)  instruction.  Further, 
it  was  reported  that  students  heavily  favored  the  self-instruction  to  the 
lecture  mode.  Only  three  of  190  courses  were  said  to  have  been  failures 
on  balance. 

Experimental  comparisons  have  also  consistently  shown  higher  student 
achievement  under  individualized  instruction  than  under  traditional  lecture 
or  lecture-discussion  methods,  according  to  a recent  comprehensive  review 
of  the  literature  by  Parsons  (undated).  Moreover,  the  experimental  litera- 
ture indicates  that  individualized,  nonlecture  courses  are  very  popular 
with  students,  even  though  usually  rated  as  more  work  than  conventional 
courses. 
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The  advantages  of  individualized  instruction  over  the  conventional 
lecture-discussion  method  are  detailed  in  two  studies.  Witters  and 
Kent  (1972)  compared  traditional  and  individualized  introductory 
psychology  and  cultural  anthropology  courses.  Students  in  the  experi- 
mental section  of  the  psychology  course  had  no  lectures  indeed,  no 
formal  class  meetings  of  any  kind.  Students  participating  in  the 
individualized  anthropology  course  met  as  a group  once  a week  to  view 
movies.  Results  showed  significantly  higher  achievement,  especially 
for  students  with  low  cumulative  grade-point  averages,  in  the  nonlecture 
sections  of  both  courses  in  each  of  two  semesters.  Students  in  experi- 
mental sections  consistently  reported  greater  confidence  in  their 
mastery  of  course  material.  In  both  semesters  of  the  psychology  course 
and  In  one  semester  of  the  anthropology  course,  students  whose  instruc- 
tion was  personalized  rated  the  course  as  significantly  more  enjoyable 
than  their  counterparts  in  the  lecture-discussion  sections.  Paden  and 
Moyer  (1972)  compared  several  different  methods  for  teaching  the  first 
course  in  economics.  Most  noteworthy  was  the  fact  that  students  who 
received  the  standard  course  minus  the  lectures  achieved  as  much  as 
those  required  to  attend  lectures. 

Miniseminars 


It  is  perhaps  a portent  that  Fred  Keller's  (1968)  seminal  paper  on 
personalized  instruction  was  entitled  "Goodbye,  Teacher.  . .".  For 
now,  in  a personalized  course,  the  professor  is  no  longer  a lecturer. 

He  is  no  longer  a teacher  of  any  kind.  He  is  called  a manager.  And 
that  he  is,  not  only  in  the  Skinnerian  sense,  as  the  arranger  of  rein- 
forcement, but  in  the  ordinary  sense  as  well. 

The  CMS  employs  teachers  as  teachers,  under  conditions  that  make 
scholarly  Insight  and  human  qualities  relevant.  Seminars  are  offered 
in  which  class  size  is  kept  low  (less  than  20  students) . Only  students 
who  have  mastered  required  background  work  are  allowed  to  register,  so 
the  discussion  is  informed. 

An  important  point  is  that  seminars  would  not  be  economically  feasible 
within  traditional  introductory  courses  at  American  universities.  The 
CMS  makes  them  possible  because  the  teacher  is  saved  from  lecturing, 
routine  review,  and  most  of  the  clerical  and  management  tasks  involved  in 
giving  and  grading  examinations,  keeping  records,  and  scheduling  tutorial 
and  group  activities. 

Seminars  meet  two  hours  a week  for  four  weeks.  To  be  eligible  to 
enroll  in  a seminar,  the  student  must  have  mastered  a specific  amount 
of  core  curriculum  by  a predetermined  registration  date.  The  seminars 
are  scheduled  in  three  waves  to  accommodate  variations  in  student  work 
rate. 
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The  CMS  handles  student  registration  for  seminars.  As  soon  as  a 
student  becomes  eligible,  he  can  sign  up  for  a seminar.  Each  seminar 
has  an  enrollment  limit  and  positions  are  filled  on  a first  come,  first 
served  basis. 

Currently  seminars  on  the  following  ten  topics  are  being  offered 
under  the  CMS  program: 

1.  Public  Policy  Toward  Big  Business 

2.  Poverty  and  Welfare  Reform 

3.  Manpower  Policy  in  the  U.  S. 

4.  Economic  Analysis  of  Inflation 

5.  Causes  of  the  Industrial  Revolution 

6.  Nixonomics  and  the  Aftermath 

7.  Consumerism 

8.  Unionism 

9.  Economics  of  Transportation 

10.  Taxation 

To  date,  seven  seminars  have  been  completed  and  the  student  ratings 
are  encouraging.  On  a six-point  scale  ranging  from  "excellent"  to 
"very  poor,"  the  modal  value  for  four  of  the  seminars  was  "very  good,"  and 
for  the  other  three,  "good."  Five  of  the  seven  seminars  were  rated  "more 
interesting"  than  other  full-semester  courses  that  the  students  had  taken, 
and  two  seminars  were  rated  "as  interesting"  as  other  courses.  The  work 
load  in  all  seminars  was  rated  "reasonable,"  the  midpoint  of  a five-point 
scale  between  "excessively  heavy"  and  "very  light." 

Remedial  and  Tutorial  Instruction 


Absolutely  essential  for  an  individualized,  self-paced  instructional 
system  is  effective  back-up  instruction  when  the  student  achieves  poorly 
and  fails  to  earn  a good  grade.  Students  receive  grades  on  three  hourly 
examinations,  on  seminar  performance,  and  the  final  examination.  Students 
can  retake  alternate  forms  of  the  hourly  examinations  and  seminars  if 
their  performance  was  below  their  expectations.  Permission  to  retake  the 
test  must  be  given  by  a teaching  assistant  (who  prescribes  work  to  remedy 
deficiencies)  if  the  student  scored  below  a C on  the  exam.  It  must  be 
emphasized  that  reference  here  is  to  remedial  procedures  to  be  provided 
after  poor  performance  on  a graded  examination.  The  study-management 
component  also  provides  simple  contingencies  when  a student  does  not  pass 
the  ungraded  quiz  over  a short  reading  assignment.  In  other  words,  CMS 
has  components  to  deal  with  poor  performance  on  both  micro-  and  macroscales. 

The  CMS  provides  diagnostic  information  which  is  essential  for  remedia- 
tion. Good  diagnoses  depend  upon  information  about  difficult  aspects  of 
the  subject  matter  and  about  the  misconceptions  of  individual  students. 

With  respect  to  the  former,  item  analyses  of  the  data  collected  in  the 
initial  runs  of  CAISMS  have  provided  a virtual  blueprint  of  the  difficult 
sections  of  the  textbook.  Six  special  adjunct  lectures  with  supporting 
worksheets  are  being  used  this  semester  to  help  students  learn  those 
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difficult  sections  in  the  text.  In  addition,  reviews  are  being 
developed  which  entail  quizzing  the  student  over  difficult  sections  in 
the  text.  If  a student  misses  an  item,  he  will  be  redirected  to  the 
text  and  provided  with  a clue  to  the  correct  answer.  He  will  then  be 
tested  again,  using  a parallel  form  of  the  item. 

With  respect  to  the  diagnosis  of  the  difficulties  encountered  by 
an  individual  student,  useful  diagnostic  information  is  a by-product  of 
the  SMS  and  the  on-line  examinations.  All  items  presented  to  the  students 
are  coded  according  to  the  content  assessed.  Since  a topic-by-topic 
profile  of  the  student's  performance  can  be  displayed  at  a PLATO  terminal, 
the  student  and  his  teacher  can  see  where  the  problems  are.  The  ability 
to  pinpoint  problem  areas  and  retrieve  the  specific  items  that  the 
student  missed  on  an  examination  is  a useful  tutoring  aid,  since  "a  few 
minutes  of  instruction  which  gets  precisely  at  the  student's  difficulty 
can  be  worth  more  than  hours  of  general  review"  (R.  Anderson  & Faust, 
1973).  The  teaching  staff  has  signed  on  at  terminals  118  times  per 
week,  and  has  averaged  12  terminal  hours  per  week  in  remedial  work. 

On-Line  Mastery  Achievement  Testing 

After  the  student  has  made  himself  eligible  for  taking  an  on-line 
test,  he  may  reserve  a time  for  testing  by  using  a reservation  routine. 

The  CMS  keeps  track  of  a student's  eligibility  and  will  not  allow  him 
to  reserve  a terminal  until  he  has  completed  the  proper  reading  assign- 
ments. The  exams  are  offered  Sunday  afternoons  and  three  evenings  dur- 
ing the  week.  The  original  intention  was  to  have  a completely  free 
access  testing  facility.  However,  this  plan  was  modified  because  the 
terminals  were  inaccessible  during  the  day.  Also,  it  was  discovered 
that  students  needed  an  economics  proctor  to  clarify  test  items  and 
deal  with  hardware  malfunctions. 

Test  items  are  displayed  to  the  students  in  a random  order.  There 
is  a spacer  page  between  items  that  tells  how  many  items  remain  in  the 
test  and  how  much  time  is  left  to  complete  them.  The  student  cannot  go 
back  to  a previous  item  to  change  an  answer  once  he  has  seen  a new  one. 

At  the  end  of  the  test,  performance  is  judged  and  test  statistics  are 
displayed.  The  student  is  shown  the  number  of  items  correct,  percent 
correct,  his  letter  grade,  and  a profile  of  performance  across  textbook 
chapters.  He  is  also  shown  his  cumulative  grade  in  the  course,  which 
includes  the  results  of  the  just-completed  test.  If  the  test  performance 
is  not  satisfactory  to  the  student,  he  can  choose  to  retake  it,  using  a 
different  form. 

To  date,  approximately  500  examinations  have  been  administered.  Test 
data  are  very  encouraging.  Students  as  a group  are  doing  as  well  on 
these  tests  as  students  who  took  them  in  previous  years  under  conventional 
lecture  procedures. 

When  students  choose  to  retake  a test  (an  alternate  form) , their 
performance  increases.  About  25%  elect  to  retake.  Two  procedures  are 
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thought  to  reduce  the  number  of  retakes.  First,  students  cannot 
continue  their  CAISMS  work  while  waiting  to  take  a test,  and,  second, 
the  most  recent  test  grade  is  the  one  that  counts.  If  a student  fails 
to  prepare  adequately  for  a test  retake,  he  may  have  to  live  with  a 
lower  grade  than  he  received  the  first  time. 

Examinat ion  Review  Items 


To  assist  the  student  in  preparing  for  the  graded  exams,  a large 
pool  of  review  items  has  been  prepared.  The  items  are  similar  in  for- 
mat and  language  to  those  on  the  test.  The  student  has  free  access  to 
this  pool  after  he  has  become  eligible  to  take  the  exam.  A random 
sample  of  ten  items  is  displayed.  Feedback  is  given  following  each 
answer,  and  performance  is  summarized  after  all  ten  items  have  been 
completed.  Students  may  try  as  many  samples  of  items  as  they  wish. 

Most  students  make  extensive  use  of  this  review  facility.  (Neither  the 
CAISMS  items  nor  the  review  items  appear  on  the  graded  exams.) 

Message  Exchange 

Another  valuable  feature  of  CMS  is  the  communication  network.  The 
students  were  assembled  only  once  as  a group  (on  the  first  day  of  class) . 
However,  as  the  semester  progressed,  they  required  general  interest 
information  on  such  matters  as  changes  in  course  procedures  and  reminders 
about  deadline  dates,  and  special  interest  information  such  as  the 
reason  why  a student  missed  a particular  item.  An  on-line  message  system 
was  evolved  in  which  students  could  direct  questions  to  the  staff  by  typ- 
ing them  on  the  terminal  keyboard.  The  staff  reviews  the  messages  during 
office  hours  and  directs  responses  to  individual  students,  or,  occa- 
sionally, to  all  students.  A student  receives  both  individual  and  the 
"all-student"  messages  when  he  signs  on  the  system  again.  Students 
leave  approximately  eight  messages  during  each  24-hour  period. 


IMPLEMENTATION  OF  THE  COURSE-MANAGEMENT  SYSTEM 

During  the  first  class  session  of  the  economics  course,  360  students 
received  instructions  on  how  the  Course-Management  System  works,  how  and 
where  to  sign  onto  the  PLATO  system,  and  specific  topical  content  areas 
covered  in  the  course.  Specific  questions  about  the  course  were  referred 
to  discussion  sessions  of  35  students,  each  conducted  for  the  first  two 
weeks  by  the  teaching  assistants.  Meanwhile,  students  began  within  a few 
hours  after  the  initial  meeting  to  sign  onto  the  system  and  start  the 
self-paced  study.  Figure  1 shows  the  rate  at  which  students  began  work- 
ing. 
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Figure  1.  Student  progress  in  computer-managed  course 


Approximately  200  students  had  taken  the  first  Study-Management 
quiz  by  the  end  of  the  first  week,  and  over  300  students  had  done  so 
by  the  end  of  the  fourth  week.  Previous  studies  on  this  population 
of  students  (T.  Anderson,  et  al.,  1974a,  1974b)  show  that  approximately 
25%  of  the  students  who  appear  for  the  first  class  session  or  who 
register  for  the  class  drop  the  course  or  change  sections  before  the 
semester  ends.  Consequently,  it  is  reasonable  to  expect  80  students 
to  drop  the  course.  To  date  approximately  50  students  have  not  started 
the  course,  and  many  others  who  have  started  are  not  likely  to  complete 
it. 


Followup  interview  data  from  80  students  who  were  well  behind  the 
course  pace  at  the  end  of  the  third  week  indicate  that  approximately 
15%  had  negative  attitudes  toward  both  the  course  and  CMS,  and  planned 
to  drop  the  course.  Slightly  more  than  18%  had  neutral  or  positive 
attitudes  toward  CMS,  but  had  to  drop  the  course  for  other  reasons.  A 
large  group,  approximately  58%,  had  positive  or  neutral  attitudes  to- 
ward CMS  and  the  course,  but  had  chosen  not  to  work  on  the  course  early 
in  the  semester. 

Student  Procedures  for  Using  CMS 

Students  are  required  to  spend  a majority  of  their  time  studying  the 
text  in  conjunction  with  the  computer  terminals.  Consequently,  they 
study  in  one  of  the  two  resource  centers  where  terminals  are  available. 
One  center  has  six  terminals  and  is  available  from  0800  to  midnight  on 
weekdays,  and  from  1300  to  midnight  on  Sundays.  Another  site  with  30 
terminals  is  available  to  students  evenings  and  weekends  to  accommodate 
the  large  number  who  study  at  these  times. 

Students  use  a terminal  for  approximately  10  to  20  minutes  during 
each  hour  of  studying.  Consequently,  from  three  to  six  students  can 
use  one  terminal  in  an  hour.  Most  of  their  computer  time  is  spent  in 
the  study-management  phase  of  the  course,  although  they  are  also  involved 
in  other  computer-assisted  activities.  Figure  2 shows  the  typical  pat- 
tern of  the  student-computer  interaction.  Options  1 through  8 are  always 
available  to  the  student,  while  others  like  9 through  12  are  contingent 
on  successful  progress  in  the  course  and  are  not  available  at  all  times. 
For  example,  a student  cannot  sign  up  for  a seminar  until  he  has  passed 
an  on-line  test,  which  he  cannot  take  until  he  has  completed  all  of  the 
study  assignments  required  for  the  test  and  reserved  a terminal  for 
testing. 
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STUDENT  SIGNS  ON  TERMINAL 
(NAME,  COURSE,  PASSWORD) 
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THE  STUDENT  CAN  CHOOSE  TO: 

1.  Read  course  information  and  procedures 

2.  Inspect  reports  of  study  management  progress  and  test 

performance 

3.  Inspect  current  distribution  of  grades  in  course 

4.  Compose  messages  to  instructors 

5.  Read  messages  from  instructors 

6.  Reserve  a terminal  for  future  study  sessions 

7.  Continue  in  the  study  management  mode 

8.  Inspect  seminar  rosters 

9.  Enroll  in  a seminar 

10.  Review  "practice"  items  in  preparation  for  an  on-line 

test  (OLT) 

11.  Reserve  a terminal  to  take  an  OLT 

12.  Take  an  OLT 


Figure  2.  Index  of  student  options  in  the  Course-Management  System 
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Staff  Procedures  for  Using  CMS 


The  system  is  managed  by  a teaching  staff,  and  a technical  staff. 

The  teaching  staff  includes  the  lead  professor,  five  graduate  teaching 
assistants,  and  six  undergraduate  tutors.  The  technical  staff  consists 
of  a project  director,  a project  manager,  and  four  graduate  research 
assistants. 

The  technical  staff  is  large,  since  the  system  is  still  in  a develop- 
mental phase.  Evaluation  and  research  data  are  being  collected  in  con- 
junction with  the  on-going  program.  When  the  system  reaches  a stable 
state,  the  staff  requirements  can  be  reduced  to  five  teaching  and  one 
technical  member,  plus  the  undergraduate  tutors. 

The  staff  can  sign  onto  the  system  at  any  terminal.  Figure  3 is  a 
schematic  of  the  staff-computer  interaction,  including  three  of  the  more 
important  reasons  for  signing  on.  The  staff  has  access  to  a large  amount 
of  current  information,  and  the  opportunity  to  intervene  and  alter  pro- 
cedural variables.  Some  course  policies,  (e.g.,  concerning  when  students 
are  eligible  to  take  on-line  exams  and  seminars)  cannot  be  changed  via 
the  staff  program.  These  policies  are  implemented  in  software  packages 
and  only  the  programmers  can  change  them.  Also,  all  of  the  courseware 
(questions,  answers,  and  assignments)  are  in  computer  language  and  can- 
not be  altered  via  the  staff  program. 


CONCLUSION 

The  Course-Management  System  was  developed  and  tried  out  in  an 
introductory  economics  course  with  an  enrollment  of  360  students.  A 
comparison  with  student  performance  data  from  prior  years  shows  that 
present  students  have  learned  as  well  and  are  favorably  disposed  toward 
the  CMS. 
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THE  STAFF  CAN  CHOOSE  TO: 

1.  Read  course  information  and  procedures 

2.  Read  messages  from  students 

3.  Compose  messages  to  individual  students  and  to  all  students 

4.  Inspect  and  alter  seminar  rosters 

5.  Add  and  delete  seminar  offerings 

6.  Inspect  and  alter  sign-up  rosters  for  on-line  tests  (OLTs) 

7.  Determine  the  site  and  number  of  terminals  to  be  used  during 

each  testing  session 

8.  Inspect  current  distributions  of  grades  in  course  from  all 

students  or  certain  subsets  of  students 

9.  Inspect  reports  of  study  management  progress  and  test  per- 

formance of  individual  students,  or  summation  reports  from 
groups  of  students 

10.  Add  or  alter  students1  seminar  and  final  exam  grades 

11.  View  any  question  used  in  the  study  management  part  of  the 

course 

12.  View  any  question  used  in  the  OLTs 

i 

STAFF  LEAVES  TERMINAL 


Figure  3.  Index  of  staff  options  in  the  Course-Management  System 
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